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Abstract. We study a triangular frustrated antiferromagnetic Heisenberg model 
with nearest-neighbor interaction J\ and third-nearest-neighbor interactions J3 by 
means of Schwinger-boson mean-field theory. It is shown that an incommensurate 
phase exists in a finite region in the parameter space for an antiferromagnetic J 3 while 
Ji can be either positive or negtaive. A detailed solution is presented to disclose the 
main features of this incommensurate phase. A gapless dispersion of quasiparticles 
leads to the intrinsic T 2 -law of specific heat. The local magnetization is significantly 
reduced by quantum fluctuations (for S — 1 case, a local magnetization is estimated 
as m = (Si) « 0.6223). The magnetic susceptibility is linear in temperature at low 
temperatures. We address possible relevance of these results to the low-temperature 
properties of NiGa2S4. From a careful analysis of the incommensurate spin wave 
vector, the interaction parameters for NiGa2S4 are estimated as, J\ ~ — 3.8755K and 
J3 w 14.0628K, in order to account for the experimental data. 
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1. Introduction 

In two-dimensional (2D) antiferromagnets, it was proposed the "geometrical frustration" 
may enhance thequantum spin fluctuation and suppress the magnetic order to form a 
spin liquid [X\. In this context the triangular- and kagome-related lattices are studied 
extensively to seek quantum spin liquid |2j. It turns out that the triangular lattice 
antiferromagnet with nearest-neighbor (NN) coupling exhibits 120° magnetic order [3], 
while the kagome lattice antiferromagnet is still a controversial topic for intriguing 
exploring [I]. People resort to other interactions, such as longer range and multiple- 
spin exchange ones, to realize quantum spin liquid [2]. Experimental evidences in favor 
of this long-predicted spin-liquid state have emerged in recent years [5] , although many 
aspects are still elusive. The spin disorder at low temperatures found in the compound 
NiGa2S4, in which Ni spins (S = 1) forms a stack of triangular lattices, aroused much 
attention [SI El El E] - The crystal structure of the material is highly 2D, since inter- 
layer interactions are quite weak. Intriguing low-temperature properties of this material 
include T 2 -law of specific heat, incommensurate short-range spin correlation, and lack 
of divergent behavior of the magnetic susceptibility. A dominant third-nearest-neighbor 
(3rd-NN) antiferromagnetic (AFM) interaction J3 could produce the incommensurate 
phase in a rough picture: four sublattices will form commensurate 120° magnetic order 
separately if the NN interaction J\ is zero, and the system will be driven into an 
incommensurate order if Ji is gradually switched on. A first-principle calculation by 
Mazin (TO] suggests a large 3rd-NN interaction J3 and a negligible 2nd-NN interaction. 
J 3 is confirmed to be AFM, but the sign of J\ has not yet been identified [10]. The 
classical spin version of this model was studied in a Monte-Carlo simulation [llj . which 
provides some helpful informations such as the incommensurability. Up to now, the 
quantum spin version of this model has not yet been studied very well. Besides the 
sign of Ji, many aspects of this model, either in agreement or disagreement with the 
experiment of NiGa2S4, need further clarification and treatments. In this paper we 
focus on the low-temperature properties of the quantum spin model and intend to make 
a contribution to this topic. 

The Schwinger-boson mean-field theory (SBMFT) provides a reliable description 
for both quantum ordered and disordered antiferromagnets based on the picture of the 
resonant valence-bond (RVB) state [TJ [121 IS] • As a merit, it does not prescribe any prior 
order for the ground state in advance, which should emerge naturally if the Schwinger 
bosons condense in the lowest energy states. For the Heisenberg antiferromagnets with 
NN couplings at zero temperature, it successfully captures the (n, n) magnetic order 
on the square lattice and the 120° magnetic order on triangular lattice respectively 
[T2| [TBI HH [15]. By means of SBMFT, we will show that the J1-J3 model falls into an 
incommensurate order phase at zero temperature for an AFM J3 and either a FM J\ 
or an AFM J x . By analyzing the incommensurate spin wave vector, we find that the 
NN interaction J\ should in the FM region to obtain an appropriate incommensurate 
phase. We also show that the T 2 -law of specific heat is an intrinsic feature of this 
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phase, the magnetic susceptibility is linear in temperature, and the local magnetization 
is significantly reduced by quantum fluctuations. We address possible relevance of these 
results to low temperature properties of NiGa 2 S4. Our results suggests that the J1-J3 
model is an essential part of the minimal model for NiGa2S4. In the following, we first 
present a formalism of the SBMFT scheme for the J\- J3 model, then solve the mean-field 
equations numerically and calculate relevant quantities. Finally we discuss the physical 
meanings of the results. 



2. The Schwinger-boson mean-field theory 

The J1-J3 model on the triangular lattice reads 

H = J 1 Si-S^Js £ S.-S,., (1) 

(ij)eNN (i'j')e3rd-NN 

We set J3 > 0, but J\ can be either AFM or FM. In the Schwinger-boson representation 
for the spin operators, = a\bi,S^ = b\a^Sf = \a\di — b\bij /2 with [aj, a]] = 

ij, we decompose the NN and 3rd-NN interactions as[16j 

JiSj ■ Sj = J\ : FijFij : —JiA^Aij, (2) 

JsSv.S^-Jsn+.TI.y, (3) 

with Fij = (a\aj + b\bj)/2, A {j = (a.j&j - bidj)/2, and II-y = (oyfy - b i 'a j >)/2. 
Correspondingly, we introduce three competing mean fields, F = (Fij), A = —i (Aij), 
and II = — i and apply the Hartree-Fock decompositions for the interactions. A 

Lagrangian multiplier A is also introduced to impose the constraint on the Schwinger 
bosons, +A ^aja.j + b\bi — 2S^ . After performing the Fourier's transform, the 
effective Hamiltonian can be written in a compact form, 

#e// = £4M(k)0 k + £ o , (4) 
k 

where k = (a k , 6 k , a_ k , 6_ k ), M (k) = e (k) a ® a + A (k) a y (g) a y , e (k) = A — 
JiFE*cosfeW,A(k) = JiA^sinA;^ + J 3 n £ 5 sin 2fcW, e = 3AT A (-JiF 2 + J^ 2 + 
J3II 2 ) — AaA (2S + 1), and <g> means the Kronecker product, o"o is a 2 x 2 unit matrix, 
z) are Pauli matrices, = k x ,k x /2 + \^3k y /2, — k x /2 + \/3k y /2 for 



cr„ s (a = x 



a o yu. a/, y, x-y cue J. ciu.11 inauiiura, rv n x , rv x / ^ \ V unj y/ n X / " > V <->n/^ 

5 = 1,2,3 respectively. The Matsubara Green's function are defined as, 

G(k,r) = -(T T ^(r)4V (5) 



where r is the imaginary time and k (t) = e rHeff <f)] Si e~ THeff . All physical quantities can 
be expressed in terms of the matrix elements of the Green's function. 

The Matsubara Green's function in Matsubara frequency uo n = 2mr/(3 (n is an 
integer for bosons) can be worked out as 

. N tu n a z O o- - e (k) (T O o- + A (k) a, ® 
G(k, z^ n ) = -^-2 — y - v -. (6) 

(lUn) -UJ Z k 
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From the poles of the Matsubara Green's function, the two degenerate spectra of the 
quasi-particles can be readily read out, 

w(k)= Ve 2 (k)-A2 (k). (7) 

The mean-field equations are established by the constraint and the introduced mean 
fields. We omit the details and only present the results here, 

^£( 1 + 2n B [u (k)]) l£> = 2S + 1, (8a) 

k 

1 / r e (k) cos k^ s > , , s 

_£(i + aBB[uW ])-UJ^. — = F , m 

1 / r „ mn A(k)Y;,sinA;W „ , s 

_^(l + 2 nB [a,(k)])^i^ = A (8o) 



6 _X> + 2, B Mk)])^L = n, 



k 



where % [a; (k)] = [e^MAsT — l] is the Bose-Einstein distribution function. In 
the thermodynamical limit — ► oo, the momentum sum is replaced by an integral, 
Xlk — > (XI Abz) J d 2 k, A B z = 8n 2 /^/3. If the Schwinger bosons condensation 
occurs at k*, a condensation term should be extracted in the momentum summation of 
the first equation, Eq. flHaj) . 



2S+l = p + [ ^ (1 + 2n fl [« (k)]) ( 9 ) 
J A B z uj[k) 

where the density of condensates 

e° = xX il + 2nB[ulk ' )]) ^Wy (10) 

k* 

Our numerical solution demonstrates the condensation occurs at zero temperature for 
spin S > Sc with Sc < 0.172. Thus we will count condensations in later discussions of 
this paper. The condensation terms in the next three mean-field equations, Eq. (|8fel) - 
(l8dj) . should also be extracted carefully. It is noticeable the per site ground state energy 
can be simplified by utilizing the mean-field equations, 



^(l>(k)+,o) 



EolN A = — ( > ^ (k) + ) = -3./,U 2 /••-} 3./:.ll- ( 1 1; 

3. The incommensurate phase solution 

The mean-field equations are solved numerically at zero temperature. For our 
purpose, we set S — 1 in the calculation in order to compare the result with the 
related experiment, although the qualitative conclusion is spin-independent, but the 
quantitative results vary with the values of spin. One fact that should be noticed is 
that the mean fields F and A could not exist simultaneously [TH [17] , so the number of 
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Figure 1. (Color online) The gapless spectrum with nodal points. To compare with 
the experiment, we choose parameter J1/J3 = —0.2756, so that the gapless nodal 
points occur at k* = ±(fc*/2, \/3k* /2) with k* = 0.1587T. The blue hexagon denotes 
the first Brillouin zone. See more details in the text. 



mean-field equations can be reduced from 4 to 3 in both J\ > and J\ < regions. In 
the two regions, we found the system falls into the incommensurate phases with gapless 
excitations. 

The quasiparticle's spectra become gapless at the nodal points, say k* = (k*, k*) = 
±(/c*/2, y3&*/2) (e.g. see Fig. 1). Near the nodal points, the spectrum is linear in 
|k-k*L 



w(k) » a|k-k*| +0 (|k-k*| 2 ) 



(12) 



At a finite temperature, a gapful spectrum will develop asymptotically as A gap = 
Cie _C2//T with constants c\ and C2, which coincides with the Mermin- Wagner theorem 
|13j . The incommensurate order at zero temperature of the system is signalled by the 
divergence in the static spin structure factor, 



xs* (qj 



^i[P(k + q)g(k)-i?(k + q) J R(k)] 



N A ^ 2 

k 



(13) 



where P (k) = [e (k) /u (k) + 1] /2,Q (k) = [e (k) /uo (k) - 1] /2,R (k) = A (k) / [2uo (k)]. 
Because the spectra is gapless at k*, uj (k*) = 0, xs z (q) becomes divergent at q* = 2k* 
(see Fig. 2), 



xs' (q 



(14) 



as it is proportional to the number of lattice sites N\. The local magnetization will be 
reduced significantly due to strong quantum fluctuations, 



Figure 2. (Color online) The zero-temperature static spin structure factor at the 
parameter J1/J3 = —0.2756. The blue hexagon denotes the first Brillouin zone. The 
divergent peaks located at q* = 2k* indicate an incommensurate order. 



The important difference between the regions of J\ > and J\ < is the nodal 
point's momentum k* G [7r/6,7r/3] for J x > and k* G [0, 7r/6] for J x < regions, 
respectively. In the limit of J1/J3 — > 00, k* — > 7r/3, the solution reproduces 120° spin 
order correctly. While below the critical value J1/J3 ~ —3.71, the system becomes 
a saturated ferromagnet, where the linear expansion, Eq. ( fl2l) . will be replaced by a 
parabolic form u> (k) ~ /3 (k — k*) 2 . The plots of k* versus J1/J3 and a versus J1/J3 
are shown in Fig. 3. 

The incommensurate spin wave vector observed in NiGa2S4 is k* = 0.1587T < 7r/6. 
From this data we estimate that J1/J3 ~ —0.2756 from Fig. 3, which is slight different 
from the value —0.20 in Ref.[6j, i.e. we have a considerable FM J\. Thus we can exclude 
the possibility of AFM J\ [10J. The local magnetization at this point evaluated by Eq. 
( TT5|) is 0.6223, (not S = 1), while the experimental data of NiGa 2 S 4 suggest a larger 
value, 0.75(8) [6]. 

The nodal structure of the spectra, Eq. (Tl2l) . leads to a linear density-of-states 
(DOS) in energy E, 

D (E) — 2 S(E — u (k)) w —zE. (16) 

k 

where the factor 2 comes from the degeneracy of the quasiparticle spectra. As a result, 
a T 2 -law of specific heat follows apparently, 

Cv/Na „ ^wi T ,, (17) 

where ( (3) = 1.202. If one supposes that the T 2 -law of specific heat of NiGa2S4 is 
ascribed to the gapless incommensurate phase, a numerical estimation, J\ w — 3.8755K 
and J3 ~ 14.0628K, could be obtained. This result is reasonable compared to the 
experimental estimation J3 = 30K [18J. 




-4 -2 2 4 6 8 10 



J\lh 

Figure 3. (Color online) (a) The condensation term po versus J1/J3. (b) The 
magnitude of the nodal point's momentum of the spectrum k* versus J1/J3. In the 
limit J1/J3 — > +00, the result reproduces the 120° commensurate spin order correctly. 
The incommensurate spin wave vector observed in NiGa2S4, k* = 0.1587T, lies in the 
Ji < region. Please see more details in the text, (c) The coefficient a in Eq. (fT2"|) 
versus J1/J3. 



4. Discussions 

Before ending this paper, we point out that the zero-field susceptibility for this 
incommensurate phase is linear in temperature, 

Xm/N a « T. (18) 

Using the parameters noted above, we find that it is \m ~ 2.77 x 10~ 4 T(emu/mole), 
which is not in agreenment with the experimental data of NiGa2S4, xm ~ A + BT with 
A m 0.009(emu/mole) and B w below 10K [6]. The Monte-Carlo study also shows 
the classical version of this model only produce a single peak in the specific heat [11]. 
These facts indicate that the model in Eq. ([I]) may not account for all mysteries in 
NiGa2S4. Thus, the solution shows the model Eq. (OQ) with AFM J3 and FM J\ has 
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captured the main features for an incommensurate correlation in NiGa2S4, but it is still 
oversimplified as the minimal model for all low temperature properties of NiGa2S4. A 
biquadratic interaction might be a good candidate for reproducing a finite susceptibility 
at zero temperature. In the absence of the 3rd-NN interactions, a biquadratic term can 
induce a quadrupolar order and totally suppress the spin order. The T 2 -law of specific 
heat is also intact when quadrupolar order sets in |19] [201 [21] . It will be interesting to see 
how the incommensurate spin correlation be influenced by the biquadratic interactions. 
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